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ABSTRACT 

The  logic  of  u technique  for  employing  technician  "confidence  that  a defect 
exists"  for  maximizing  the  probability  of  malfunction  recognition  is  described.  The 
technique  is  based  on  and  drawn  from  parallel  thinking  in  signal  detection  theory. 
Operator  characteristic  curves  are  derived  for  a variety  of  distributions  of  confi- 
dence. " Continuous  and  discrete  distributions  of  "confidence"  are  considered  as 
well  as  single  and  double  criterion  levels.  The  implications  of  the  work  for  train- 
ing and  posttraining  performance  evaluation  are  pointed  out. 
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IN  I'ltQIHV  I'lON 

Previous  i'iM‘;iri'h,  completed  by  \pplicd  Psychological  Services,  has  attempted 
to  sot  into  focus  methods  for  estimating  personnel  subsystem  reliability.  1'hese  studies, 
drawn  ! rout  and  based  on  a serves  of  investigations  into  "personnel  psychophysical"  re- 
lationships, sought  to  establish  techniques  for  quantitatively  assessing  the  probability 
of  successfully  performing  a given  avionic  maintenance  act.  In  the  first  of  these  studies 
into  personnel  subsystem  reliability  (Siegel  \ Pfeiffer.  P.HUll.tho  utility  of  the  ratio  of 
effective  to  effective  phis  ineffective  performances  was  investigated  as  the  basic  in- 
gredient for  personnel  subsystem  reliability  estimation.  Measures  of  effective  and  iti- 
etleclvve  performance  were  obtained  for  Fleet  avionic  maintenance  personnel  through 
magnitude  estimation  methods.  I he  measures  were  anchored  to  a number  of  avionic 
jot)  dimensions  isolated  through  a multidimensional  scaling  analysis  of  the  avionics 
maintenance  job.  V\  hen  c ompounded  through  techniques  which  arc  analogous  t o those 
employed  in  traditional  electronic  reliability  prediction,  the  evoked  numerics  were 
found  to  discriminate  between  squadrons  known  to  he  of  different  competency  by  other 
criteria  and  to  discriminate  within  squadrons  in  accordance  with  reasonable  expecta- 
tion. Siegel  and  Pfeiffer  concluded  that: 

The  obtained  avionic  personnel  subsystem  (reliability! 
indices  seem  to  be  useful  for  posttraining  performance  apprais- 
al, personnel  placement,  and  squadron  evaluative  purposes. 

Because  of  the  reasonableness  of  the  results,  the  ease  of  employment,  and  the 
general  utility  of  the  approach  employed  bv  Siegel  and  Pfeiffer,  Siegel  and  Miehle  ( 1 '.HIT! 
extended  the  work  to  consider  a number  of  circumstances  not  originally  considered.  Vd- 
ditionally,  a measure  ot  effectiveness,  as  separate  from  "reliability,  was  derived. 


1 In-  efloctivoness  calculation  considered  not  only  the  reliability  of  the  individual  tech- 
nicians performing  the  maintenance  acts,  but  also  elapsed  time,  amount  of  manpower 
employed,  and  job  activity  repetition.  It  was  contended  that  the  technique  useful  for: 

l.  quantitative  comparison  ol  the  elfcctiv  eness  of  different  teams 
or  individuals  who  perform  the  same  task 

-•  prediction  of  the  performance  effectiveness  of  a team  or  of  an 
individual  on  a task 

.1.  derivation  of  training  requirements 

4.  optimization  of  personnel  assignments  and  maintenance  pro- 
cedures 

evaluating  the  design  of  new  systems  or  alternative  designs 
of  the  same  system 

Additionally,  Siegel  and  Miehle  developed  nomographs  and  tables  to  simplify  the  com- 
putational aspects  of  the  procedures  they  described. 

I’u  r p n s e of  Pri-ifiil  He  port 

rile  previous  studies  consider  only  peripherally  the  trouble-shooting  aspect  of 
avionic  maintenance.  Within  the  technique  described,  the  job  activity,  "electro  cogni- 
tion, is  considered  to  encompass  a host  of  mental  acts  involved  in  the  maintenance  and 
trouble-shooting  of  avionic  equipment.  Yet,  it  is  known  that  malfunction  recognition 
represents  a particularly  troublesome  aspect  of  the  maintenance  procedure,  whether 
periodic  inspections  or  malfunction  diagnosis  and  location  arc  involved,  flood  com- 
ponents are  often  erroneously  replaced  and  marginal  or-  defective  components  over- 
looked. 

\s  the  logical  extension  of  the  previous  work  in  "personnel  psvchophysics,  " it 
seemed  reasonable  to  apply  the  psychophysical  developments  of  signal  recognition  theory 


3. 


to  the  malfunction  recognition  issue.  The  central  problem  then  becomes  that  of  demon- 
strating a quantitative  method  for  determining  the  probability  that  an  avionic  technician 
will  say  that:  (1)  a defect  is  present  when  a defect  is  in  fact  present,  and  (2)  a defect  is 
pr*t  sent  when  in  f«ict  no  defect  is  present. 

I he  methods  to  be  described  in  subsequent  chapters  of  this  report  are  analogous 
to  those  used  by  Swets  and  his  coworkers  (cf.  , 1964).  In  the  present  report,  the  confi- 
dence that  a defect  exists  replaces  the  perceived  intensity  of  a signal  in  their  work. 
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CHAPTER  II 


METHODS 

In  certain  avionic  maintenance  activities,  satisfactory  performance  may  be  said 
to  occur  when  action  is  taken  when  it  should  be  taken,  and  no  action  is  taken  when  no 
action  is  warranted.  Examples  of  this  occur  during  routine  equipment  inspection,  fault 
location  and  isolation,  and  malfunction  correction.  Let  us  consider  all  of  these  under 
the  generic  classification  of  defect  recognition. 

Hr  f r r t Hr  c o gn  i I i » n 

Let  x be  a measure  of  the  maintenance  technician's  confidence  that  a defect  ex- 
ists, with  0 < x < 1.  For  the  situation  in  which  the  technician  has  complete  confidence 
that  a defect  exists,  x = 1.  When  the  technician  has  complete  confidence  that  no  defect 
exists,  x = 0.  Intermediate  confidence  values  may  exist  because  of  incomplete  informa- 
tion, conflicting  information,  inadequate  knowledge  about  the  equipment  system,  etc. 

Due  to  this  uncertainty,  a technician  may  have  different  values  of  confidence  on 
different  occasions,  even  though  the  situations  presented  to  him  are  the  same.  This 
variation  is  representable  by  a confidence  distribution. 

When  there  is  a defect,  confidence  that  a defect  exists  is  high  (x  is  near  1)  most 
of  the  time.  When  there  is  actually  no  defect,  in  most  cases  x is  low. 


(i. 

This  probability  is  the  area  under  the  rurve  of  f( v)  to  the  left  of  x k.  The  distribution 
function  is  fix)  J f(u)du  and  is  the  area  under  the  curve  of  f(x)  to  tile  left  of  x.  l’lie 
probability  tliat  a x • b is  1^ [a  • x bj  Jbf(x)dx  K(b)  - F(a). 

Note  that  K'(x)  t(x).  Suppose  that  a technician  will  report  a defect  (will  say 

"yes")  wlu'ii  his  confidence  that  there  is  defect  is  greater  than  or  equal  to  some  sub- 
jective criterion  level,  c,  i.  e.  , x c.  Then  lie  will  also  report  that  .1  defect  is  not 
present  (will  say  "no")  when  x e. 

For  either  a defect  or  no  defect,  the  probability  that  a technician  will  say  "yes" 
is  !’rfx  * >']  Jj  f(x)dx.  When  there  actually  is  a defect,  then  P [x  'cl  is  the  prob- 
ability of  a correct  response;  call  this  l*  [yes  I 111.  When  Ihere  is  no  defect,  tin'll 
lyx'el  is  the  probability  of  an  incorrect  response  (false  alarm);  call  this  1*  [yes  ||V|. 

I'r  11 1111I1  i I i t \ nt  Ui*  1 1*  i'  1 III'  i1 11  xit  i t 1 11  n 

for  either  a delect  or  no  defect,  the  probability  that  a technician  will  say  "no"  is 
^ c I x v 1 .1  t(x)dx.  When  there  actually  is  a defect,  then  P^  [x  c|  is  the  probability 

of  an  incorrect  response  (failure  to  detect).  Cal!  this  P [uo|D].  When  there  is  no  de- 
lect, then  l't  [x  c)  is  the  probability  of  correct  response.  Call  this  l>  [uo|D'|. 

bet  the  frequency  function  when  defect  exists  be  Ij  and  for  no  defect  be  f,, . 

Then:  P [yes  |D]  J 1 f,  (x)  dx,  P [yes  1 1)']  ./ 1 to  <x)dx,  PfnolD]  f'f,  (x)dx,  and 

i 1 c 1 ‘ il 

I*r  [no  I !>'  I Jo  fo  ( x)  d x. 

The  probability  that  the  subject  says  "yes"  and  there  is  a defect  is;  P |y»sAl)| 

P,  [yes 'Dll’,  [I)|.  The  probability  that  the  answer  is  "no"  and  there  is  no  defect  is 
l’r  [no A IV 1 l’r  fno|lv)  p [D'[. 


I 


) 





I'lii-  probability  ot  a eorrret  response  ( of  i-orrivt  responses),  K,  is; 

K I*  | y«*s  \ I »1  t t*  IiioAD'I 

r r 

l’r  |y«  M 1 1) 1 1 ’s.l I » I » l‘r  | no  [ l > ’ | I*  ( I ) ’ | 

I’  1 1 > | /''  r,  (x)dx  t I’  |l)'|  I ‘ f,  txldx.  (n 

r l-  I'  ' (1 

Tin-  probability  of  on  ineorreet  response  is; 

l'  [vi  s A LV  | t I*  [no Al) I 

r ' r 

1’,  [yes  1 1 > ' J l'(  [O'J  t |>  [no|!)|  I ’ 1 1)  ] 

l*  [l)|  Jo  t'i  tx)dx  t l1  1 1) 1 1 J 1 f0  (xldx.  (2) 

r r v 

Ihi-  sum  ol  tlu'si-  probabilities  (eorrert  .mil  liii-ori'ci'l  response)  is  unity. 
I'Ih-si-  ri-l.itionships  ran  bo  alternatively  represented  as  mdieated  m Kxliilnt  l 

l)  IV 


I’xbibit  1 

l-'xllibit  I ran  hr  nmsidrrrd  as  a V'rmi  diagram  in  whirl)  tin-  larar  lrrt. tuple 
rrpres ruts  thr  universal  set  ol'  all  lopiral  possibilities.  The  eolumn  labeled  O repri 
sents  the  set  of  situations  in  whieh  a defert  exists.  I’he  row  labeled  "Yes"  represent 
the  set  situations  in  whieh  "yes"  was  the  response.  The  eells  represent  mterseetion 


H. 


<>!  the  sets  represented  by  rows  and  columns.  Their  measures  are  tin-  joint  prob- 
abilities, whereas  the  mt-asurrs  ot  tin-  rows  ami  > oluinns  arc  tin-  marginal  j>rol>- 
abl  llties.  Note  that  only  if  independence  exists  docs  I*  (yes  A 1)  | ! * (yes  | |>  ||>|. 

I Ills  situation  Is  a special  case  of  the  formulas  gi  \ cn  earlier. 

H*-  | i tilt  i | i | \ II II  (I  (l|i  | i an  m Ion  f i lie  nee  (rile  lion  I rod 

Reliability  K (or  the  probability  of  correct  response)  is  a function  of  1*  [1)|  and 
tlu  rout idtMU'i*  distribution; 


-• 


•<  1’  tl)|J  t'l  ( X > d X * 1*  (1)'|  J f0  (x)dx 

I C V 0 


r 


dlt 


••’or  maximum  K.  we  set  — 0 to  find  the  optimum  value  of 

is  constant.  The  implicit  equation; 


\ ■ mine  1 ’ 1 1 ) I 

r 


dll 


— -i>r  [nif,(c)  t i>f  [i)'i  r0(e)  o 


gives  optimum  value(s)  of  e. 


f.  (e) 

1 ’ r 

n;i 

fjr) 

I’r 

1 ' 


Suppose  that,  in  the  Long  run,  for  an  unchanging  situation,  the  fraction  of  time 


a defect  occurs  is  r.  Then  i*r[I) | t r and  l‘t.|l)' | ; 1 - r,  so  that  4 


file) 


1 - r. 


I o ( >' ) 


For 


example,  if  a defect  occurs  21)";,  of  the  time 


1 - 


1 ‘■2  -8  , 

” — — — s.  1 hen,  tor  maxi- 


mum reliability,  the  technician  should  (and  probably  will)  make  c such  that 

fo(e) 


As  example  1,  assume  the  graphs  of  fu  and  l|  as  shown  in  figure  3. 


. 


I'lum-c  2 I. in. Ml-  lr.'i|uni.  \ functions,  with  (f,  > and  without  (f0)  defect 

(first  example) 

Note  that  till-  vertical  scale  has  been  adjusted  so  that  thr  arra  under  earll 

, ur\r  is  equal  to  1.  Here.  f0(x)  2 - 2x.  and  l|(x)  2x,  so  that  . 

1 0 (>  ' ) 2 - 2c 

For  maxi  mum  reliability,  for  l’r(l)|  .2,  — — — 4,  or.  . tl. 

* 2"2i* 

In  this  rase,  the  terhllirian  should  re|n»rt  a defect  when  his  confidence  value  I. 
. ,‘t  or  above.  For  anvl’r(l)|  r, 

2r  l - r 

2-2r  r * 

solving  tor  , gives;  r l - r.  When  only  a few  defects  exist,  r is  low  and  eon 
sequent  ly  e is  large,  i.e  , the  subject  will  or  should  sal  yes"  only  when  lie  is  verv 
sure  that  a defect  exists. 

\s  another  example,  consider  Figure  4.  rile  area  under  each  curve  is  still  1 


Figure  -I  I. inear  frequency  functions,  with  (fit  and  without  (f,.t  defect 

(second  example) 


f0(x)  5-12.  5x  and  f^x)  12.  5x  -7.5 


10. 


On  tin-  basis  of  tin-  distribution  shown,  wlu-n  a defect  exists,  the  teehnieian 
always  has  a i-onfidenee  level  of  . (>  or  greater.  When  no  defeet  exists,  the  teehnieian 
always  has  a eonfidi-nce  of  .-1  or  less.  Therefore,  if  his  threshold  or  eriUeal  value  e is 
between  .4  and  . t>,  he  will  always  be  eorreet  (K  1).  This  should  be  independent  of 
1’rfD],  I ,et  us  compute  R for  .4  c • .0. 

K Pr[D]J*f,(x)dx  + PrlU'l  Jofo(x)dx 
Pr[Dl  (1)  + Pj.lD’1  (0  1 

Ife  < . 4,  the  teehnieian  will  sometimes  report  a defeet  when  there  is  none.  If  c . (>, 
the  subject  will  sometimes  fail  to  notice  a defect. 

Consider  the  equation  for  optimum  c: 

I'l  (e)  _ 1 - r 12.  5e  - 7.  5 1 - r 

fo(e)  r °l  5 - 12.5c  r 

When  solved  for  e,  this  gives; 

C = . 4 + . 2r. 

As  r varies  from  0 to  1,  c varies  from  .4  to  . (>.  In  this  example,  it  turns  out 
that  all  values  of  e in  that  interval  are  optimum. 

l.et  us  now  calculate  the  reliability,  K,  for  the  example  in  which  the  curves 
intersected  (first  example); 

K - r J ' 2xdx  + (1  - r)  J o (2  - 2x)dx 
= i'[x*  ]j,  + (l-r)[2x  - x*  ]o 

r(l  - cJ ) + (1  - r)[2c  - c2  - 0] 


for  optimum  defect  recognition  (c 


1 * r)  tins  becomes 


R r2  - r + 1 
rn 


l{  is  thp 
m 


maximum  value  of 


reliability  for  the  given  confidence  frequency  functions 


mui  a given  r.  See  figure  5.  The  graph  is  a parabola. 


figure  f>  Maximum  reliability  vs.  relative  frequency 
of  defeet  for  filia  tion  in  figure  3 


for  r .3  (optimum  e . U), 

R . 04  - . 2 t 1 . 04 

m 

The  minimum  value  of  R (.  75)  occurs  when  r 4-  When  r 0 or  1 R 1 
I'liis  relationship  is,  in  a sense,  in  aeeord  with  information  theory.  Uncertainty  is 
highest  for  equiprobable  events  as  when  Pr[Dl  !’r[l)'l  (or  r 4>-  Then,  it  is  reason 
abb-  to  expect  least  reliability  l{  for  r — . for  R .5,  the  information  is  11  Sp.logy 
log2  + • log  2 = t t ! 1 bit.  for  r .2,  the  information  is  . 2 log  — + a lor  — 

. 46  + . 36  . 72  bits.  No  attempt  is  made  at  the  present  to  relate  information  to  reli 

ability  in  a quantitative  manner. 


Opr  r n ting  C.htiriir  I e r i t i r Cnnr 

rn  signal  detection  theory,  receiver  operating  characteristic  (ROO  curves 
are  drawn  to  show  the  relationships  between  I’rjyes  |S|  and  fr  [yes  | S 1 1 where  S stands 


for  .i  signal  being  present.  An  analogous  type  of  operating  characteristic  curve  van 
1>«-  drawn  for  tin-  present  data  For  the  first  example; 

‘‘rives  1 » ' | J1  f0(x> dx  j1  (_»  - 2x)dx  2x  - x2  I1 

C C 1 C 

2 - 1 - (2e  - c2 ) 1 - 2c  + c*  ( 1 - c)2 

Pr[iuiO)  = j‘  fi(x)dx  j'  2xd\  x2  |L  e2 

W e have  already  found  that; 

Pr[yeslD|  = J1  I-!  (x)dx  - 1 - c"  and 

c 

l*r[nolD'|  */*  fo(x)dx  2o  - c2  . 

Note  that  Pr[yes  |t|  f !’r[no'l)|  ( 1 - e2  ) t e2  1 and 

I’rfyes  D ' 1 f Pr[no|D'1  ( 1 - c)2  +<2e-c2)  1. 

! sing  corresponding  values  of  I’rfves  l)j  unci  Pr[  ves  1 TV  | for  various  \ alues  of  e,  the 
operating  characteristic  curve  van  he  obtained . Table  1 gives  such  values. 

Table  1 

Conditional  1 ' robabi It  1 1 es  for  Various  Values  of  Criterion  l.evel 

Pr[yes  | L> ' ] Pr[yes  L)| 


c 

(1  - e)2 

1 - c 

0 

1 

1 

•) 

. 04 

. 90 

. 4 

. 36 

. 84 

. 0 

. Ill 

. 04 

. U 

. 04 

. 30 

1.0 

0 

0 

\ plot  of  the  data  of  Table  1 is  presented  as  Figure  6 and  in  Figure  1 1 as  a 1. 
(The  similarity  in  the  form  of  this  curve  and  that  of  the  receiver  operating  character- 
istic curves  of  signal  detection  theory  is  self  evident.  We  have  deliberately  called 


our  curves  operating  characteristic  curves  to  accentuate  the  similaritv.1 


I 


»] 


Figure  (i  Operating  characteristic  culwe  for  Fxample  1 

Su‘"h  eurve  (Figure  6)  is  interpretable  in  several  ways.  First,  if  a technician 
performance  tails  below  tin-  points  of  the  curve,  he  is  not  performing  malfunction  di- 
agnosis  correctly.  If  his  performance  falls  along  the  transverse  line  shown,  he  is  per 
forming  at  the  ehanee  level.  Here,  either  training  or  equipment  modification  (auto- 
matic test  equipment)  might  be  indicated.  Finally,  for  a fixed  value  of  c.  the  ordinate 
ot  a point  on  the  curve  gives  the  probability  of  a malfunction  being  detected,  given  that 
a maltunction  is  present;  the  abcissa  gives  the  corresponding  probability  that  a mal- 
tunction  will  be  reported  when  none  exists,  m fact. 

Kxpnnent  iul  Con  f i tl.n.r  Disiributii.n 

Hit  vis  assume  that  technician  contidonco  possesses  an  exponential  distributiiui. 
I.xponential  confidence  t requeney  curves  are  shown  in  Figure  7. 


Figure  7 Frequency  functions,  with  Hi'  and  without  (f„)  defect 


14. 


-x 


The  function  for  no  defect  is  of  the  form  fo(x)  = ke  . The  area,  under  the 
curve  must  be  unity  and  accordingly: 


J1 ke"Xdx  = 1 
0 


ke  X I'  = - k(e  - 1)  = 1 
o 


k = 


e - 1 


= 1.582 


, , , e -x  e 

fn  (x)  = : e 


1-x 


e - 1 e - 1 


. 582e 


1-x 


The  function  when  there*  is  a defect  is; 


f!  (X)  = ; = . 582eX. 


e - 1 


The  various  conditional  probabilities  are; 


Pr[yes  |D]  = /^dx  = 


1 x 1 1 1 . c.  e-e 

— re  | = — r (e-e  ) = 

e- 1 1 c e- 1 e- 1 


c 


1-e 


e-i 


1-e"1 


l -x 


Pr[yes  | D ' J = /J.JjTT'dx  = ‘ — 


1 l -x  , 1 
e-Te  lc 


e° -e1 "c 


e1  "c  - 1 


e-  1 


e - 1 


Pr[no'D]  = 1 - Pr[yeslT]  = dx 


x 

e ,c 


eC  - 1 


e- 1 1 o e- 1 


Prtno'D'l  = 1 -Pr[yeslT'l  = J"C- — dx  = 


i -x 


l -x 

e ,c 


e-  1 


e-1  ( 


1-e 
1 - e" i 


and 


R = Pr[D]  Pr[yes  | D ] + Pr[D ']  Pr[no  | D '] 


,l-eC_1  A l-e"C, 

= r<rr^)+  (1-rHYT^r) 


< l|i  t i in  u m I ii  n f i il  i-  ii  • ■■  I i-  i i-  | f I \ p <■  n «•  n I i u I ( n s ■■  s 


If  oil.'  wishes  to  determine  the  eonfidenee  level  that  the  technician  should  adopt 
u>  order  to  obtain  a maximum  value  of  K,  the  technician  should  adjust  his  e value  so 
that: 

ft  (c)  l-i- 

to(e)  r 


■-  1 


i ~e 

1- — r> 


c - 1 1 -r 

r 


2c- 1 ln(—  i 
1* 


2c  l+lR(  — ) 
r 


■ 1 i / 1 ~ >'» 

. f In  ( ) 

r 


Since  0 - e < 1,  the  above  formula  for  e holds  only  when: 


•+1 


>+  1 


or  . < r < .731. 


When  r 


. !?(»!> , e 1 , and  when  r 


e 

T7T 


' 31,  e 0 for  maximum  K.  For 


these  values  ot  r and  c,  K — — .731.  When  r c remains  1 and  K beconn 

‘‘*(*1  l*t  I » 


to  \ 


1-r.  When  r r . c remains  0 and  H becomes  K r. 

t’M  in 

Hu*  plot  ot  Ktn  versus  r is  shown  in  Figure  8,  where  the  curve  from  \ 

7Tt['  -2 


— us  given  by  Htn 


% r(l-r) 


. At  r . 5,  R 


. b Jo. 


Figure  U Maximum  reliability  curve  for  exponential  frequency  functions 


I .a>!r  gives  values  i)l  1 ’ r[yes  1 1)|  anil  l’r[ves  |)  | for  drawing  th « ■ operating 
characteristic  curve. 


Fat) le  : 

Conditional  Probabilities  for  Various  Values  of  Criterion  Level 
tor  I xponential  Frequency  Function 

c l'r(ve.s  1 ■> ' | l‘r[ye.s  1J| 


l) 

1.  000 

1 . 000 

•) 

. 7 13 

. 872 

. 4 

. 47  0 

.713 

. 0 

. 280 

. 522 

. 8 

. 120 

. 280 

1.  0 

0 

0 

The  plot  of  Table  2 is  shown  in  Figure  1 1 as  "exp.  " 

\ u r mu  I t t> ii  r i tl  ** ii  «-  ••  H i s l r i liu  t i u u 

I’he  ease  in  whieh  eonfidenee  is  normally  distributed  may  be  similarly 
developed.  Normal  eonfidenee  frequencx  eurves  (>'  1'  are  shown  in  Figure  0. 


Figure  0 Normal  frequency  function,  with  (t'i  ) and  without  (f0  ) defect 


When  no  defect  exists: 


fo(x) 


k - — 

e 2 


\ 2" 


/'  e 2 dx  = 1 


o 


k(.  34134)=  1 

1 


k = 


. 34134 


f,(x)  = 2.9296 


2. 9296 

_ x^_ 

e 2 


\ 2tt 


(x-ir 


f i ( x ) = 2.9296 


e 2 


V 2w 


Pr[yesjD]  = 2.  9296 


‘ _ (x- 1) 

) — dx 

J c v 2tt 


when  x = c,  u = c-1 
x = 1,  u = 0 


° iL 

e 2 


• _ 

Pr[yes  I D]  = 2.9296  - — — 

J c-  1 V 2tt 


let  u = x - 1 
du  = dx 


du  = 2.9296 


t-c  _^2 

e 2 


= 2.9296 


V 2tt 

yc  .i. 

e"  2 


I. 


du 


2tt 


/ o v'Tir 


Pr[no  I D] 


= 9 


Pr[yes  I D ']  = 2.9296 


Pr[no  | D ' ] = 2.  9296 


i 

.929 6 I - 

Ji-c 

i 


V 2tt 


e 2 


dx 


dx 


c V 2tt 


C 2 

X 


V 2tt 


dx 


r. 

fe.. 

[:! 

7 

» 


The  reliability 


bp  l i mum  ( » n I'  i 

Kor  a r 


Since  0 

When  r = . 3775 
of  r and  c,  K 

m 

When  r > . 6225 


18. 


is: 


t)286 


i ”C 


It 


-dx  + ( 1 


V 2tt 


-r)  J -i_± 

J V 2* 


-dx 


ill- mi-  !,»■  \ i-  I fur  No  r in  ii  I l)i  s I r i liu  l i mi  ('.use 

elative  maximum  of  It,  the  technician  should  set  his  c value  so  that: 


fi  (c) 


fo(c) 


[ - r 
r 


(c-,r  c 


/e  2 


1 - r 
r 


e " 2 


C “ 2 


= In  (- — — ) 
r 


C -2  + 111  ( ' ' 1 


c < 1,  this  formula  holds  only  when; 
1 1 


1 + e 


+-  i 


- i 

1 + e 2 


or  .3775<r£.6225. 


, e 1,  and  when  r = .6225,  c 0 for  maximum  it.  Kor  these  values 
.6225.  When  r .3775,  c remains  1 and  R becomes  it  r 1-r. 


, c remains  0 and  It  becomes  It  = r.  Table  3 gives  It  for  values 
in  in  ° m 


‘ 1 


.. 


y 


i 


* \ 


I 


of  r between  . 3775  and  . 6225. 


in. 


N ii  r in  ii  I 


■'  is; 


W lu'ii  no 


M i s l r i Ii  ii  I i o n * i I Ii  ii  ii  \ \ h I u t‘  of  ir 

Ihf  formula  tor  the  normal  curve  with  a standard  deviation  of  any  value  of 


f(x) 


0 V JIT 


i“  Jir  . 


defect  is  present: 


>„<*>  — 

u v 2n 


. . x dx 

let  U — , du  — 
<r  it 


for  x 0,  u 0;  for  x 


_> 

— ii 

ilu  1. 

v Hit 


and 


1 

u — , so 

tl 


ible  a gives  k tor  various  values  ot  o.  As  o approaches  0 , k approaches  2. 

Table  5 

Normalizing  Constant  (k)  for  Various  Values 
of  Standard  Deviation 


2 

1 

IS  ■’ 2 2 
2.  930 

1 /2 

2.  095 

1/3 

2.  005 

1/4 

2.  000 

■ ■ 


1 

, \ 


J . 


I 


* 

t 


r 


When  a defect  is  present, 


fi  ( xl 


(x-  n2 

2 J 2 


Normal  I > ■ * i r i Itu  I i on  » i I li  J . •> 

As  an  example  of  a value  of  o different  from  unity,  take  .5. 
f0(x)  4.  190  e’Jx 


l'i  (\) 


4.  1 90 


-2(x-n 


The  confidence  frequency  eurves  are  shown  in  I*  igure  12. 


f igure  12  Normal  frequency  function  (o  . 

with  ( f x ) and  without  (f0)  defect 


This  represents  a considerable  improvement  over  « 1 shown  in  figure  !l 


.i 


MIM 


0l»  I 


S Ill'll 


Hus 


rJ 


du  + ( 1 - 


o v 2 it 


" — 

J u v2t 


du 


* ' 'I'"'  ''  Normal  Ili  *t  r i Ini  l i on  „ i | |,  <r  s 

In  the  |>r.'s.Mit  imsc,  for  a rotative  maximum  of  U,  the  technician  should  sot  . 
that : 


fi  M 1 - r 
r 


. (c-l)J 

i1  2.i » 1 -r 


reduces  to; 


1-r 


r 


ln(— 1 
r 


i 

C * 5 


in  (— ) 
r 


\ + „2 


In  ( 


r 


ini  o 0 < c < 1(  this  formula  holds  only  when; 


1'able  7 Hives  values  of  Pr(yesiD')  a.ul  Pr(yes  D)  for  various  values  of  > 


T.ible  7 


Conditional 

Probabilities  for  Various  V 
(Normal  Frequency 

’.dues  of  Criterion 
Function) 

c 

l'rfves  1 1)’  1 

Pr| yes  'll] 

0 

1 . 000 

1 . 000 

a 

. 674 

. 933 

. 4 

. :i<»6 

. 806 

. fi 

. 104 

. 604 

. 8 

. 067 

. 326 

l.  0 

0 

0 

I ,evel 


These  are  plotted  in  Figure  11  .is  j 5. 

rul  Straight  line  « .» n f * «1  «•  n . e l»i  *t  r i bu  H «>n 

|,et  us  mow  consider  the  situation  in  which  a more  general  straight  line  tr. 
quency  function  of  confidence  exists.  This  is  shown  in  Figure  14. 


igure  14  1 linear  frequency  function,  with  <f,»  and  without  (f0>  defect 


The  x and  v intercepts  of  y 
•) 

trary  and  b - - . 


f0(x)  occur  at  a and  b respectively,  where  a is  arbi 


* / 


L 


26. 


fu(x)  = -(1--) 


cl  U 

2.2  1 


fi(x)  = — x + -<!--) 


Pr[yes  Dj  = 21-!-1-1-)  - (— -)2 


Pr[no'Dl  = 1 - 2(- — -)  + (—  7 Vy 


Pr[yes  ! D ' 1 = (1  - -) 


£,2 


2c  c2 


PrfnolD'l  = — - - — 
a a2 


R = r S 


For  a given  value  of  r,  the  maximum  R occurs  when- 


c = a + ( 1 - 2a)r  and  is  given  by: 


Km  = — |2ra[(l -a)  - (1  - 2a)r]  - r[(  1 - a)  - ( 1 - 2a)r] 2 
+ (1 -r)[a  + (1 -2a)r][a  - (1 -2a)r]  I . 


As  an  example,  let  a = . 75; 


8 4 

fo(x)  = j (1  - — x) 


r , . _ 32  8 

h U)  - —X  - - 


The  optimum  c level  exists  when; 


3 1 

c = — - — r 

4 2 


Values;  of  K for  various  values  of  r are  given  in  Table  8.  The  graph  is 
shown  in  Figure  15. 


Table  8 


Figure  15  Maximum  reliability  curve  for  linear  frequency  function 

(a  = . 75) 

Table  !*  gives  values  of  Prfyes  lD'1  and  Prfves  ID]  for  various  values  of  c 

Table  H 

Conditional  Probabilities  for  Various  Values  of  Criterion  Level 
(Linear  Frequency  Function) 


A 


k J'1  f|  (x)dx 


k /‘[(Si  (x)  t S,  (x)  + 

0 . 


S (x)]dx 


= k [.  75  + . 5 + . 25  ] 1.5k 


1.5k  1 


k - — so: 

O 


Lot: 


Then 


fi  (x>  T7 1 -S i ( x ) + S,(x)  + S,(x)] 

.5  4 2 4 


f„(x)r  2 - t'o(x)  2 --HSi(x)  + S,(x)  + S , (x)[ 

.5  4 4 


I c ( a , b)  = 
o 


f S (x)dx 

Jb  6 


b-a  if  6<a<rb 
b-6  if  * a < 6 < b 
0 if  a < b < 6 


I’Hves  1 D]  = f fi(x)dx  ~ f [Si(x)  + Si(x)  + S>(x)]dx 

C O C 4 2 4 

*7  [ 1 1 (c,  1)  + 1 1 ( c , 1)  + I j (c,  1)] 

0 4 2 4 

1 1 

Hr[yes  IV]  = J f0(x)dx  = J ^ < _ 3 _ 


1 I 2 ) 

' > - —[Si  (x)  + Si(x)  + S,(x)]  ' dx 

\ 


2c  - -rr [I i (e,  1)  + Me,  1)  + h(c,  D] 


3 1 « 


Table  10  gives  values  of  Pr[yes|D]  and  Pr[yes|D']  which  are  plotted  in  Figure  11 
as  curve  S.  The  graph  consists  of  four  straight  line  segments  (one  being  horizontal 
and  one  vertical). 


30. 


Table  10 

Conditional  Probabilities  for  Various  Values  of  Criterion  Revel 
(Step  Function) 


c 

Pr[yes  I)1] 

Prfyes  1 D 

0 

1 

1 

. l 

. 8 

1 

. 25 

. 5 

1 

. 4 

. 3 

. 9 

. 5 

. 167 

. 833 

. 6 

. 1 

. 7 

. 75 

0 

. 5 

. 9 

0 

. 2 

1.  0 

0 

0 

Pr[no  D)  = 

JA  fi  (x)dx  = 
0 

tHT  (0,  c)  + I . (0,  c)  + 

O 4 2 

MO.  c)] 

4 

Pr[nojD’]  = 

J*~fo(x)dx  = 
0 

2c  - 4[Il(0,  c)  + 1,(0, 

0 4 2 

, e)  + li(0,  c)] 

4 

R = 

4r  [Me,  1) 

O 4 

+ Me,  l)  + Me,  l)] 

2 4 

+ ( l-r)  | 2c 

- 4 [MO,  e)  + 1,(0,  c) 

o 4 

+ M0,  c)]l  . 

< 


Simplifying  gives: 

R r + 2c(  1 - r)  - 4 [MO,  c)  + 1,(0,  c)  + I3(0,  c)l  . 

<5  4 2 4 


Due  to  the  nature  of  the  frequency  functions,  it  is  not  possible  to  solve  for 

optimum  c for  a given  r value  from  the  equation  -li-44  = . Table  11  eives 

fold  r 6 

f ( c ^ 

values  of  -1  , . and  Table  12  gives  values  of  — 

to(e)  B r 

Table  11 

Frequency  Ratio  vs.  Optimum  Criterion  Revel 
(Step  Function) 

Me) 

t: fp  ( e ) 


0 to  . 25  0 

. 25  to  . 5 .5 

. 5 to  . 75  2 

. 7 5 to  1 ® 


Table  12 


31. 


Table  of 


1 - 


(Step  Function) 

1 - r 


0 

00 

. 1 

9 

2 

4 

. 3 

2.  33 

. 4 

1.5 

. 5 

1 

. 8 

. 887 

. 7 

. 428 

. 8 

. 25 

. 3 

.111 

1.0 

0 

Suppose  r . 3.  The  value  of  — — is  2.33.  The  closest  v alue  of  is  2 

1 fowl 

and  e is  between  .5  and  .75.  To  find  the  maximum  value  of  R for  r .3,  the  value 

of  c would  have  to  be  varied  between  .5  and  .75  and  U calculated.  When  this  is  done 

K is  found  to  increase  linearly  from  . 833  to  . 35  as  c increases  from  . 5 to  . 75.  Abov 

. i o , it  drops  linearly  to  . 70  for  c 1.  Therefore,  tor  r .3,  K 87) 

in 


(•f  n era  I (.use  of  TTri|urn<'\  liinilions 

The  general  case  of  arbitrary  frequency  functions  u til  now  be  considered.  As 
an  illustration,  two  arbitrary  curves  were  drawn  with  a French  curve.  Function 
values  tor  to  and  tj  were  read  and  used  to  compute  the  areas  under  the  curves  by 
Simpson's  one-third  rule.  The  ordinates  were  then  scaled  so  that  the  area  under 
each  curve  equaled  unity,  thus  giving  frequency  curves.  These  are  shown  in  f ig- 
ure 18.  Table  13  gives  the  function  v alues. 


Table  13 


Normalized  Frequency  Function  Values  for 
the  Hypothetical  Example  of  Figure  18 


Values  of  Aq(x)  = /'  f0(x)dx  and  Ai  (x)  = f fj  (x)dx  are  given  in  Table  14  and 


plotted  in  Figure  19 


Table  14 


Areas  under  Frequency  Curves  from  0 to  x 


functi  >ris , with  (Ai  > a.nci  without  <Ao>  dofect 


I*  or  example.  Let  r .3.  I lieu  — - — — . We  must  lU’trrmiiu’  a value  < > 1 \ sueli 

r *5 

that  -p  — . This  an  In-  done  by  drawing  a template  with  two  lute;,  w ith  ,lop<  ■ 

having  ratios  of  ~ , as  in  Figure  20,  on  a transparent  sheet. 


Figure  20  Template  with  lines  having  slopes  7:2  ratio 


The  height  of  every  point  on  the  upper  line  is  ■—  the  height  of  the  corresponding  point 
directly  below  it  on  the  lower  line,  1'he  \ axis  of  this  template  is  made  coincident 
with  the  x axis  of  the  graphs  of  t\,  anil  I't  and  displaced  hori  ontallv  so  that  line  1 
intersects  fj  and  line  0 intersects  f0  for  the  same  value  of  x.  Phis  occurs  when  x 
.707  so  that  e . 707  for  r .2.  The  maximum  value  of  It  can  now  be  computed  for 
r 3. 


1 •707 

It  . 3 J ft  (xhlx  * . 7 | f(i(x)dx 

m .707  0 


1 

/ ft  (x)dx  1 - A 1 (.  70  ,')  t - . 320  . 4 HO 

• 7 07 


• 7 07 

f f0(x)dx  .929 
0 


It  . 3(.  «mn  i . 7( . 9291  . 794. 

m 


If  other  values  of  e are  chosen  (with  r fixed  at  .3),  lower  values  of  It  result. 


Table  la  illustrates  this. 


Table  la 


Ueliability  vs.  Criterion  l.evel  for 


Ti-lyes  ) l»'  | j f0(x)dx  1 - \0(e) 

c 

1 

Prfyes  111)  J f,  (x)dx  1 - A,  (r) 

c 

e Iti  gives  values  of  Pr[yesl|V]  and  Prfves  j 

Table  1(5 


Conditional  1 ’robabi  lili es  lor  Various  Values 
of  C riterion  I ,e\  el 


•) 

. 652 

, 07  ii 

4 

. 363 

. 34  1 

6 

. 151 

. 62  1 

3 

. 027 

. 340 

0 

0 

0 

'■urve  is  so  elose  to  that  labeled  a 1,  Dial  i(  was  no!  plolled. 


37. 


D i s r r etc  D i s t r i b u t ion  of  Confidence 


In  the  Naval  operational  situation,  only  a few  levels  of  individual  technician 
confidence  are  probably  identifiable.  When  only  a finite  number  of  confidence  values 
are  involved,  they  can  be  represented  as  shown  in  Figure  21  and  22. 


Figure  2i  Probability  distribution 
when  a defect  exists 


Figure  22  Probability  distribution 
when  there  is  no  defect 


The  vertical  lines  have  been  drawn  at  various  points  to  make  the  graphs  more  per- 
spicuous. When  a defect  exists,  the  probability  for  x = 1 is  highest  (in  this  illustra- 
tion, pi  (1)  = . 54).  When  there  is  no  defect,  the  probability  for  x =0  is  highest 
(po(0)  = .6).  Values  for  this  arbitrary  example  are  shown  in  Table  17. 


Table  17 


Probability  Distribution 
x Pi(x)  p0(x) 


0 

23 

49 

1 


0 

. 14 
. 32 
. 54 


. 6 
. 3 
. 1 
0 


, ’ 


r 


h 


a 


In  the  discrete  case,  p0(x)  and  pi  (x)  correspond  to  f0(x)  and  I'i(x)  respectively 


ilpo(x)  and  — pi  (x)  correspond  to  J'f0(x)dx  and  J’fj  (x)  respectively.  As  before; 


Pr[yes]  = Pr[x  » c) 

1 I 

PrfyeslD)  ^ Pi  M.  Pr[yes|D']  ^ Po(*> 

x"*c  x>c 

x c x<c 

Pr[no  | D]  = ^2  pi  (x),  Pr[no|D']  T"!  Pn(x)  . 

x = 0 x = 0 


For  the  discrete  case,  when  e is  a value  for  which  po  or  pi  is  defined,  then  it  makes 
a difference  whether  Pr[yes|  Prtx-*  c]  or  Pr(x  ^ c |.  The  two  probabilities  differ  by 
p(c). 

I x-  1 

K = pi(x)  + < 1 * r)  Po(x) 

x'i  c x = 0 


This  is  also  a maximum  when; 


Pi  (c)  (1-1) 

Po(e)  r 


Consider  an  example  which  corresponds  to  the  continuous  example  shown  in  Figure  3, 
Table  l!i  gives  the  values  of  p0  andpi.  Table  111  also  presents  the  (cumulative)  dis- 
tribution and  ratios  of  pi  to  p2 . For  r = , 8,  .25.  From  the  table,  we  see 

that  the  nearest  value  of  the  ratio  is  so  e should  lie  between  0 and  . 25. 


Table  m 

l’robabi lit v Distribution  and  Ratios 

x1  x^)|,l(x)  f\) 


Table-  10  gives  ranges  oi  optimum  e and  values  of  K for  various  values  of  i 


Table  10 

Range  of  Optimum  Criterion  Level  and  Maximum  Reliability 
vs.  Relative  Frequency  of  Defeats 


The  values  of  R were  obtained  from  Table  JO,  wliieli  gives  values  of  R foi 
m 


various  values  of  e and  r. 


Table  20 


Reliabilitv  and  Conditional  Probabilities  vs.  Criterion  Level 


Pr[yes  | D | 

= ') 


40. 
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The  operating  characteristic  curve  has  four  points:  (.  6',  1),  (.3,  .0),  (.1,  .7), 

and  (0,  .4).  The  K curve  has  a minimum  of  .8  for  r .5. 
in 

l>«»  ii  ti  1 «*  Criterion  l.eiel  Deter  lion  Model 

To  this  point,  we  have  discussed  a detection  model  with  a single  criterion 
level.  When  the  confidence,  x,  of  there  being  a defect  is  greater  than  or  equal  to 
the  criterion  level,  c,  the  subject  will  say  "yea"  (there  is  a defect).  When  x c, 
the  subject  will  say  "no.  " 

Now  consider  two  criterion  levels,  cj  and  c2 , where  ci  < c2 . If  x<  cj  , 
the  subject  will  say  "no.  " If  x » c2  , he  will  say  "yes.  " If  c!  x c2 , then  the  sub- 
ject will  make  another  attempt  to  determine  the  status  of  the  equipment.  This  con- 
tinues until  he  says  either  "yes"  or  "no.  " This  type  of  procedure  has  also  been 
called  by  Swets  (1964)  " sequential  observations.  " A third  possibility,  not  con- 
sidered here, is  that  he  may  be  allowed  only  a limited  number  of  trials.  If  not  able 
to  decide,  this  might  constitute  a third  type  of  unreliable  operation  on  the  part  of 
the  subject,  the  first  two  types  occurring  when  subject  says  "no"  with  a defect,  and 
"yes"  with  no  defect. 

Assume  that  the  curves  shown  in  Figure  23  represent  frequency  functions. 


Figure  23  Hypothetical  frequency  function,  with  (fj)  and  without  (f0)  defect 


1 


I'll*'  area  under  each  would  then  be  unity.  The  two  criterion  levels  are  represented 

1 

by  ct  and  c2  . The  area  under  a curve  f from  c-  to  1 [call  a A(c„. , 1 > ] represents 
the  probability  (pi  ) that  the  technician  will  say  "yes"  on  one  trial.  Similarly,  A(0,  c) 
is  the  probability  (pa)  of  saying  no  and  is  represented  by  the  area  under  a curve  l from 
0 to  ci  on  one  trial.  The  probability  (pj ) of  being  undecided  on  a single  trial  is  tin 
urea  under  a curve  I from  ei  to  cj . File  trials  are  considered  independent.  The 
probability  tree  in  Figure  24  shows  only  three  stages.  It  holds  for  either  a defect  (1)1 
or  no  defect  (1)'). 


l’r[yes  | 

Pr[no] 

Specifically; 


Finally,  reliability  is; 


y y 

Y 

' V"  V 

/ 

/ *2 

\ 

\ 

y 

3 \ 3^ 

V 

N N 

N 

Figure 

■ 24  Probability 

tree 

Pi  + PaPi 

+ pjpi  + |>a  pi 

~T 

Pa  + P2P3 

+ P2  P3  + p2  P3 

— 

Prfyes  jDj 

A,  (c2 

* i) 

Ai  (ca , 1 ) + 

A i ( 0 , ci) 

Pr[no  1)'] 

A0<0, 

Cl) 

Ao  (i'2  , 1 ) + 

Ao  (0,  ci) 

R 

r Pr[yes  |D] 

+ (1  - r)l 

r Ai  (t\  , 

1) 

Ai (c; , n + 

Ai(0,  c,) 

Pi  +Ps 


Pi  + P3 


U-1-1  \„(P,  ci> 

a0(c,.,  n + \„(o,  c , i 


•11’. 


1 1\<‘  expected  number  oi  trials  for  a decision  (yes  or  no),  if  up  to  n trials  urn 
allowed,  was  derived  previously  (Siegel  * Miehle,  1967)  as. 


F 

n 


n 

+ npi 


lira  Kn 
n * '' 


l 1 

1 " A(e2  , H + A(0,  v , ) 


riii.s  holds  for  both  the  cases  in  which  a defect  or  no  defect  exists.  The  ease  where 
an  unlimited  number  of  trials  are  allowed  therefore  is; 

K L + 1 ~ 1 

A i ( c j , 1)  + Aid),  d)  Ao  (c.  , 1)  + ,\u(0.  Cl)  * 


rhe  increased  reliability  obtained  by  repeated  trials  is  obtained  at  tile  expense  of 
increased  time  and  effort. 


As  an  illustration,  consider  the  linear  frequency  functions  of  Figure  a. 


A,  <02,  1) 
A i ( 0 , c, ) 
Ao(C2,  1) 


A0(0,  cj) 


ltd  d - C2  * ci ; 

R 


' i i 

/ f i ( x)dx  = f 2xdx  X'! 
cj  ' e.  C2 


1 - c-. 


Cl 


/ fo(x)dx 


i i 

f (2  - 2x)dx  = 2x  - xJ 


= 1 - 2c 2 + 02 


2x  - X-  I = 2c,  - cf 

0 


i ;? 


1 - c'  + 


2 + Cl 


(1  - r)( Jci  - cT) 

- 2C2  + C2  t 2c,  - c‘. 


1 


1 - d(ci  + C2  ) 


1 t d(o,  + c2)  - 2d 


* ''  l - d(cj  + cj ) + 1 - 2d  +d(ci  + cT) 

i:  is  plottod  as  a function  of  e — — foi'  r .3,  .5  for  d .2,  .4,  .6 

CD  * J 

(Figure  25).  R is  plotted  us  u function  of  e for  d 0,  . 2,  .4,  . (i  for  r . 3 m 
Figure  26,  and  for  r .5  in  Figure  27. 

It  can  be  seen  tliut  the  optimum  value  of  c is  .5  for  r .5  and  that  the  max- 
imum value  of  K increases  as  the  separation  (d)  between  the  criterion  levels  increase;* 
For  r .3,  the  optimum  value  of  c decreases  .is  d increases.  file  maximum  values 

(K  ) of  K increase  with  increasing  d and  are  higher  than  those  for  r .5  as  ex- 

m 

pected. 

The  curves  of  F for  d ^ .2  have  relative  minima  whose  location  shifts  to 

00 

the  left  for  increasing  d,  but  they  do  not  coincide  (except  for  r .3)  with  the  max- 
ima of  It. 

Table  21  gives  values  of  optimum  c,  R and  I t for  various  values  ot  d, 
when  r - . 3 and  . 5. 

Table  2 1 

Optimum  Average  Criterion  l,evel.  Maximum  Reliability , and 
Expected  Number  of  Trials  vs.  Separation  (d)  of  Criterion  I ,c\  < Is 


CM!  \ 1 ■ l'l  i;  II! 


■1  7 . 


DISCISSION,  SIMM  \ |;  V . \ NP  CONC  1 ,1  Slv  IN 

Hi  m’  ii  n s i on 

In  lli>'  present  report,  reliabi li ty  and  operating  i hararte  ristn  . urves  w . i . 
considered  lor  various  types  of  "confidence  of  tin  cxistem  . of  a defeel"  ill  .intui- 
tions. I'lns  is  an  adaptation  of  tin  method  used  by  rainier  and  Suets  (Idol),  where 
stimulus  is  replaced  by  confidence.  If  the  dist  nlnit ions  for  defect  and  no  defect  are 

symmetrical  about  \ ',  then  the  maximum  reliability  (H  1 is  lowest  when  a ileftvi 

m 

occurs  with  probability  of  '.  This  is  the  situation  of  maximum  uncertainty . Dor  all 

continuous  distributions  considered,  the  maximum  reliability  attainable  equals  I when 

a defect  is  never  present  (r  0)  and  when  a defect  is  always  present  (r  11.  The 

lowest  value  of  11  (at  r . 51  depends  on  the  dist  rilnit  ions.  The  smaller  the  com  - 
in 

moil  area  under  both  curves  (1)  and  ID',  the  larger  the  K values  are  (see  1'able  ’ '1. 

m 

When  the  curves  do  not  intersect,  11  is  1 for  all  values  of  r.  When  r is  low,  tin  opti 

in 

mum  criterion  level  (el  is  high,  and  when  r is  high,  v is  low.  Drom  the  point  at 

which  e first  becomes  0,  the  plot  of  II  vs.  r becomes  a straight  line  (il  rl.  Drom 

m 

the  point  where  c first  becvimes  1,  H vs.  r becomes  the  straight  line  l\  1 - r. 

Table 

Comparison  of  Common  \rea  under  f,  and  f0  I'lirve.-- 
with  Maximum  lioliahility  tor  r ,a 

Case  Area  !{,,,  for  r . 5 


Non-intersecting 

0 

l 

Straight  line,  a .75 

. 885* 

Straight  line,  a 1 

. 5 

. 75 

Normal,  o l 

. 57 

.7  15 

exponential 

. 755 

. 823 

Normal,  o 1 

. 87  8 

. 5 8 l 

4«. 


A similar  progression  holds  for  the  operating  characteristic  curves.  As  the 
common  area  under  the  distribution  curves  decreases,  the  operating  characteristic 
curves  move  closer  to  the  left  and  upper  boundaries. 

The  reliability  value'  (R)  may  be  improved  in  two  ways.  The  first  is  to  mini- 
mize or  eliminate  the  intersection  of  the  frequency  curves,  as  indicated  by  Table  22. 

When  the  curves  are  non-intersecting,  the  maximum  attainable  reliability  is  1. 

This  reliability  is  obtained  when  the  confidence  criterion  level  is  properly  set.  One 
method  for  achieving  this  level  is  to  design  the  equipment  so  that  the  existence  of  a 
defect  is  more  obvious.  This  will  separate  curves  f\  and  f0 , making  the  setting  of 
a criterion  level  less  critical.  With  non -intersecting  curves,  the  optimum  criterion 
level  is  anywhere  between  the  x intercepts  of  the  curves.  j 

With  a given  pair  of  frequency  curves,  the  reliability  may  also  be  improved 
by  training  the  technician  so  that  he  sets  his  criterion  level  towards  the  optimum 
value.  This  can  be  accomplished  by  providing  the  technician  with  feedback  of  in- 
formation about  his  performance  or  possibly  through  programmed  instructional 

ft 

techniques. 


^ mi  “« — I — | — L |_ 

49. 

Summitry  mid  I'.onc  I u s i on 

The  present  report  has  attempted  to  point  out  the  relationship  between  the 
subjective  criterion  level  that  a maintenance  technician  sets  or  adapts  for  accept- 
ing or  rejecting  the  hypothesis  that  a malfunction  exists  and  the  probability  of  his 
acceptance  or  rejection  of  the  hypothesis  being  correct.  The  logic  employed  is 
similar  to  that  used  in  signal  detection  theory,  and  operating  characteristic  curves 
were  similarly  derived.  These  may  be  interpreted  in  much  the  same  way  as  the 
receiver  operator  curves  derived  in  the  psychophysical  investigation  of  signal  de- 
tection. The  relationships  were  derived  for  a number  of  continuous  distributions 
of  confidence  and  for  the  discrete  case.  Both  the  single  and  double  criterion  in- 
stances were  considered.  It  is  believed  that  the  discrete  case  with  a single  cri- 
terion level  represents  the  most  useful  case,  from  the  point  of  view  of  application. 
While  we  have  not  attempted  to  specify  a method  for  measuring  a technician's  con- 
fidence criterion  behavior,  the  production  of  such  a change,  and  the  consequent  in- 
crease in  correct  hypothesis  acceptance  and  false  hypothesis  rejection,  does  not 
seem  unattainable. 

The  gains  to  be  derived  from  the  production  of  such  a shift  in  criterion  level 
are  considerable.  Consider  example  1 (Figure  3).  Assume  the  probability  of  a de- 
fect is  r . 1.  The  optimum  criterion  level  is  c - 1 - r .9.  At  that  level,  the 
maximum  possible  reliability  is  R r~  ■ r t 1 = .01  - . 1 + 1 .91.  Suppose  the 

technician  actually  assumes  a level  of  c - .5.  Then  his  reliability  is 

R r + 2c  - c2  - 2 re 
- . 75 


1 


i 

» 


This  is  a considerable  reduction  from  the  maximum  of  .91. 


5U. 


It  is  possible  that  a programmed  learning  technique  or  prog  ram  could  be  de- 
veloped to  allow  individual  technician  training  on  acceptance  criterion  level  setting. 
Thus,  the  acceptance  criterion  level  training  could  be  administered  quite  economi- 
cally, with  the  usual  advantages  of  programmed  learning. 

Alternatively,  the  employment  of  the  acceptance  level  setting  behavior  of 
technicians  as  a fleet  performance  criterion  seems  tenable.  Technicians  who  set 
their  criterion  of  acceptance  so  as  to  maximize  correct  hypothesis  and  rejection  are 
probably  superior  to  those  who  do  not.  Thus,  if  for  a pair  of  technicians  and  a given 
set  of  equipment  conditions,  technician  A sets  his  criterion  level  less  close  to  the 
optimum  than  technician  B,  then  technician  B may  be  said  to  be  superior.  Tech- 
niques for  performing  such  measures  should  be  readily  derivable. 

Finally,  it  was  pointed  out  that  the  employment  of  the  operating  characteristic 
curve  possesses  implications  for  decisions  regarding  the  development  of  automatic 
test  equipment.  Test  equipment  would  be  built  for  those  situations  in  which  malfunc- 
tion detection  deviates  significantly  from  the  optimum,  or  stated  alternatively,  where 
it  approaches  the  chance  level. 

In  conclusion,  it  seems  that  a method  for  establishing  the  optimum  subjective 
criterion  of  acceptance  level  has  been  established  for  the  malfunction  recognition 
situation. 
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